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(54) A system and method for detecting defects in a surface of a workpiece 



(57) A surface inspection system (1 0) is provided for 
detecting defects on a surface of a workpiece The sys- 
tem (10) inciudes a diffused light source (22) for emit- 
ting an elongated line of light (23) onto the surface of the 
workpiece (25), a movable member (26) for translating 
the workpiece (25) in relation to the light source (22), an 
imaging device (24) positioned at a vantage point such 
that the line of light {23) is within its field of observation 
for capturing two or more sets of image data represent- 
ative of a portion of trie surface of the workpiece, and a 
data structure for storing model data, where the model 
data is indicative of the spatial relationship between the 
surface of the workpiece (25) and the observation plane 



of the imaging device (24). The surface inspection sys- 
tem further inciudes an anomaly detection module (28) 
adapted to receive the image data from the Imaging 
device (24) and for identifying at least one potential sur- 
face defect in the image data, and a defect tracking 
module (30) connected to the data structure and the 
anomaly detection module (28) for tracking the potential 
surface defect from the first set of image data to the sec - 
ond set of image data by using said model data, thereby 
assessing if the potential surface defect constitutes a 
defect in the surface of the workpiece (25). 
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Description 

Background and Summary. of the Invention 

[0001 ] The present invention relates generally to an 
automated surface inspection system and, more partic- 
ularly, a system and method for detecting defects on a 
painted surface of a workpiece 

[0002] Demand for higher quality has pressed man- 
ufacturers of mass produced articles, such as automo- 
tive vehicles, to employ automated manufacturing 
techniques that were unheard of when assembly line 
manufacturing was first conceived. Today, robotic equip- 
ment is used to assemble, weld, finish, gauge and test 
manufactured articles with a much higher degree of 
quality and precision than has been heretofore possible 
Computer-aided manufacturing techniques allow 
designers to graphically conceptualize and design a 
new product on a computer workstation and the auto- 
mated manufacturing process ensures that the design 
is faithfully carried out precisely according to specifica- 
tion 

[0003] Quality control is also an important compo- 
nent of the automated manufacturing process For 
instance, rather than employing human inspectors, 
automated surface inspection systems are used to per- 
form repetitive visual inspection of a workpiece in order 
to detect flaws in the surface of a workpiece 
[0004] An imaging device is typically used in an 
automated surface inspection system to capture frames 
of image data representative of a portion of the surface 
of the workpiece. To determine what constitutes a sur- 
face defect, the surface inspection system compares 
the location of a potential surface defect from frame to 
frame. In the conventional case, the surface inspection 
system assumes that the inspection surface fs substan- 
tially pfanar; and thus the potential surface defect is 
moving at the same or proportional rate (in relation to 
the imaging device) at which the workpiece is being 
translated by the movable member, However, for con- 
toured inspection surfaces, the potential surface defect 
does not travel at the same rate in relation to the imag- 
ing device as the velocity of the movable member As a 
result, conventional surface inspection systems are 
unable to accurately compare the location of a potential 
defect between frames, and thus may be unable to iden- 
tify surface defects in contoured inspection surfaces, 
[0005] Therefore, it is desirable to provide an auto- 
mated system and method for detecting defects in a 
contoured surface of a workpiece To the extent that the 
inspection surface is painted, St is also desirable that the 
surface inspection system distinguish surface ffaws 
caused by dirt, pinholes or scratches from the surface 
roughness of the paint (i e , "orange peel") on the work- 
piece. It would further be advantageous if such a sys- 
tem were able to characterize the defects by size so that 
oniy those body parts having defects larger than a cer- 
tain threshold size would need to be rejected by the 



inspection process 
Summary of the Invention 

5 [0006] In accordance with the present invention, a 
surface inspection system is provided for detecting 
defects on a surface of a workpiece. The surface 
inspection system Includes a diffused light source for 
emitting an elongated line of light onto the surface of the 
10 workpiece, a movable member for translating the work- 
piece in relation to the fight source, an Imaging device 
positioned at a vantage point such that the line of light is 
within its field of observation for capturing two or more 
sets of image data representative of a portion of the sur- 
fs face of the workpiece. and a data structure for storing 
model data which is indicative of the spatial relationship 
between the surface of the workpiece and the observa- 
tion plane of the imaging device The surface inspection 
system further includes an anomaly detection module 
2o for identifying at least one potential surface defect in the 
image data and a defect tracking module connected to 
the data structure and the anomaly detection module for 
tracking the potential surface defect from the first set of 
Image data to the second set of image data, thereby 
25 assessing if the potential surface defect constitutes a 
defect fn the surface of the workpiece 
[0007] For a more complete understanding of the 
invention, its objects and advantages, reference may be 
had to the following specification and to the accompany- 
30 mg drawings 

Brief Description of the D r awin gs 

[0000] 

35 

Figure 1 is a perspective view of a surface inspec- 
tion station on an automotive assembly iine in 
accordance with the present invention; 

40 Figure 2 is a block diagram of the basic compo^ 
nents of the surface inspection system of the 
present invention; 

Figures 3A-3B illustrates image processing opera- 
45 tions which are appfied by the anomaly detection 
subsystem of the present invention; 

Figures 4A and 4B are block diagrams illustrating 
the operations performed by the defect tracking 
so subsystem in accordance with the present inven- 
tion; 

Figure 5 is a diagram showing how the velocity of a 
potential surface defect in relation to the imaging 
55 device varies from the velocity of the conveyor sys- 
tem; 

Figure 6 illustrates an exemplary 30 facet model for 
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an automotive vehicle body in accordance with the 
present invention; 

Figure 7 is a diagram showing an exemplary 
numeric assignment for each pixel location near the s 
predicted pixel for a potential surface defect which 
is then used to determine a tracking strength for 
that potential surface defect in accordance with the 
present invention; and 

10 

Figure 8 is a diagram illustrating how identified sur- 
face defects are mapped back to a mode J of the 
vehicle body in accordance with the present inven- 
tion 

15 

Description of the Preferred Embodiment 

[0009] An exemplary surface inspection system of 
the type commonly employed in vehicle assembly lines 
is shown in Figure 1 The surface inspection system 10 20 
includes a conveyer system 12 for carrying vehicle bod- 
ies 14 through various assembly stations on the assem- 
bly line A mounting frame 16 surrounds the conveyor 
system 12 and provides a mounting position for at least 
two light sources 22 In addition, the mounting frame 1 6 25 
provides a plurality of other mounting positions for a 
series of Imaging devices 24 Communication cables, 
which are not specifically shown in Fig 1, couple the 
light sources 22 and the imaging devices 24 to one or 
more computing devices 18, Although the invention is 30 
not limited to automotive applications, an exemplary use 
for the surface inspection system of the present inven- 
tion would be in an automotive assembly plant 
[001 Oj Figure 2 illustrates the basic components 
associated with the surface inspection system 10 in 35 
accordance wfth the present invention The surface 
inspection system 10 generally includes two diffused 
light sources 22, one or more imaging devices 24, a 
movable member 26 an anomaly detection subsystem 
28, and a defect tracking subsystem 30. 40 
[0011] Two light sources 22 are used to emit two 
elongated lines of light 23 onto the surface of a work- 
piece 2S Preferably, each light source 22 is a fluores- 
cent lamp diffused through a thin sheet of diffusing film 
An exemplary light source is manufactured by Plastic 45 
Films Company it is also envisioned that a baffle (not 
shown) may be positioned between each of the light 
sources 22 Although two light sources are presently 
preferred, this is not intended as a limitation of the 
invention, On the contrary, one or more fight sources so 
may be suitably used 

[0012] At least one imaging device 24 Is used to 
capture image data representative of a portion of the 
surface of the workpiece The imaging device 24 is posi • 
tionable at a vantage point such that the fine of light 23 55 
is within the device's field of observation It should be 
noted that the length of the light source 22 is such that 
the field of observation of the imaging device 24 is fully 



illuminated by the light source 22. An exemplary imag- 
ing device 24 may be one of a variety of CCD cameras 
manufactured by Pulnix Corporation While the follow - 
ing description is provided with reference to one imag- 
ing device, it is readily understood that a plurality of 
imaging devices may be used to Inspect the same or dif- 
ferent portions of the surface of the workpiece . 
[0013] Generally, the intensity of light source along 
with the imaging device and its lens setting are chosen 
so that surface defects such as dirt, craters, solvent 
pops, and scratches create contrast in the captured 
image which is greater than the surface roughness of 
the paint on the workpiece (commonly referred to as 
"orange peel"). The light intensity gradient from the 
elongated lines of light is such that it provides a means 
for emphasizing the defect contrast relative to the back- 
ground Thus, potenfja! surface defects are detected by 
the light reflected back to and captured by the imaging 
device 24 Since the surface roughness of the paint is 
highly no n -rep eatable and varies from surface to sur- 
face on a vehicle body (e g , It being generally higher on 
a vertical surface than on a horizontal surface), the 
anomaly detection subsystem 28 provides an adaptive 
technique that de-emphasizes the image distortions 
caused by the surface roughness of the paint 
[001 43 If the surface of the workpiece is painted with 
a high reflectivity color (eg, white paint), then the inten- 
sity of the light source 22 may saturate the surface of 
the workpiece such that the imaging device 24 Is unable 
detect surface defects from the reflected light. One 
skilled in the art wiil readily recognize that the surface 
inspection system 1 0 is operative to dim or brighten the 
light sources 22 based on the color of the inspected sur- 
face. The intensity of the light source is determined 
through empirical testing for each color of the inspected 
surface which may be inspected by the surface inspec- 
tion surface 10. 

[0015] In operation, the workpiece 25 is coupled to 
a movable member 26 which translates the workpiece 
25 in relation to the light sources 22 More specifically, 
the movable member 26 travels in a direction substan- 
tially perpendicular to the lines of fight from the light 
sources 22 As a result, the inspection surface of the 
workpiece passes through the lines of light such that the 
imaging device 24 is able to capture image data repre- 
sentative of a portion of the surface of the workpiece In 
addition, the movable member 26 may travel at varying 
rates of speed., As will be apparent to one skilled in the 
art, the rate of speed may be adjusted so that the imag- 
ing device 24 is abie to capture more or less image data 
for a particular portion of the workpiece as it moves 
through the lines of light from the light sources 22. 
Although a conveyor system Is presently preferred for 
use as the movable member 26, this is not intended as 
a limitation of the broader aspects of the present inven- 
tion On the contrary, other types of movable members 
may be used in conjunction with the present invention. 
[0016] An anomaly detection subsystem 28 and a 
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defect tracking subsystem 30 reside on one or more of 
the computing devices (not shown) associated with the 
surface inspection system 10 As wilf be more fully 
explamed below, the anomaly detection subsystem 28 
Is adapted to receive image data from the imaging 5 
device 24 and operative to identify potential surface 
defects in the image data; whereas the defect tracking 
subsystem 30 is operative to track the potential surface 
defects, thereby assessing if any one potential surface 
defect constitutes a defect in the surface of the work- w 
piece. One skilled in the art will readily recognize that 
the anomaly detection subsystem 28 and the defect 
tracking subsystem 30 may be implemented using com- 
mercially available hardware and software components 
such as those available from Matrox Electronics Sys- 75 
terns Ltd. 

[0017] A more detailed description of a preferred 
embodiment of the surface inspection system 10 Is pro- 
vided in relation to Figures 3-6 In order to Identify 
potential surface defects, numerous sets of image data 20 
are received and processed by the anomaly detection 
subsystem 28 For instance, a first set of image data 32 
and a second set of image data 34 are received by the 
anomaly detection subsystem 28 in Figure 3A. Various 
image processing techniques are then applied to each 25 
pair of image frames 

[0018] First, an absolute subtraction 36 occurs 
between the first image frame 32 and the second image 
frame 34. This operation removes background noise 
from the imaged surface that are stationary relative to 30 
the imaging device. The resulting image frame 38 is 
indicative of the differences between the two sets of 
image data Absolute subtraction is preferred because it 
enhances all of the anomalies encapsuiated in both sets 
of image data, 35 
[0019] Next, the resulting image frame 3S under- 
goes a thresholding function 40 This operation reduces 
noise resulting from the subtraction operation. Moreo- 
ver; the thresholding function 40 converts the integer 
representation of the image data into a binary represent 40 
tation of the image data In other words, rather than rep- 
resenting the pixel data as an integer between 0 and 
255 f the pixel data is stored as either a 0 oM This con- 
version to a binary image frame reduces the data by 
eight fold, and thus increases the speed of the remain - 45 
ing image processing operations The resulting binary 
image frame is shown at 42 

[0020] "Orange peer is randomly varying structure 
noise that exists on a painted surface. "Orange peel" is 
part of any painting process on a meta! surface and Is so 
caused by the viscosity of the paint. Rather than soften- 
ing the sharp edges of the lines of light from the fight 
sources to minimize the effects of "orange peel" in the 
image data, the anomaiy detection subsystem 28 fur- 
ther includes a filtering process for removing the 55 
"orange peel" anomalies found in the image data As wiif 
be apparent to one skilled in the art, this filtering proc- 
ess is particularly applicable to painted inspection sur- 



faces, whereas the remainder of the present invention 
applies to any bare metal inspection surfaces 
[0021] The orange peel" filtering process begins 
with a copy of the first set of image data 32 and the sec- 
ond set of image data 34 as shown in Figure 3B First, 
an edge detection operation 52 as is well known in the 
art is used to identify the edges of the lines of light in 
each set of image data The resulting Images are shown 
at 54 and 56, As previously explained, a thresholding 
function 58 is applied to each of these images 54 and 56 
to reduce noise as weil reduce the amount of image 
data. As will be apparent to one skilfed in the art, the 
selection of thresholding is such that the resultant 
image from step 58 contains predominantly light edges, 
but not defects that are farther from the edges of the 
Jines of light 

[0022] Next, the binary image data frames are 
dilated using a dilation operation 60 as is well known in 
the art. In this case the dilation operation enhances the 
thickness of the detected edges of the lines of lights, 
The dilated images are shown at 62 and 64 Each of the 
above operations are performed on both sets of image 
data as shown in Figure 3B , Each set of image data 62 
and 64 are combined using an OR operation The "OR" 
operation 66 ensures that the "orange peel" anomalies 
from each set of image data are represented in the 
resulting composite image data. The resulting compos- 
ite image data is shown at 68 

[0023] Referring to Figure 3C, this composite image 
data 68 is inverted (using a "NOT" operation) 70 to gen- 
erate an orange peel mask 72 The orange peel mask 
72 predominantly represents the "orange peel* on the 
painted surface of the workpiece and can be applied as 
a filter to the. binary image frame 42 To do so, the 
orange pee! mask 72 Is combined with the binary image 
frame 42 using an "AND" operation 74 In this way, the 
"orange peel" is rempved from the image data. The 
resultant image frame 76 identifies only potential sur- 
face defects in the workpiece and can be used as input 
to the defect tracking subsystem 30, . 
[0024] Generally, the defect tracking subsystem 30 
receives potential surface defect information from the 
anomaly detection subsystem 1 8 and tracks the posi- 
tion of each potential surface defect as it moves within, 
the reference frame of the imaging device If the position 
of the potential surface defect is accurately predicted, 
then the potential defect may be classified as a defect in 
the surface of the workpiece, On the other hand, if the 
position of the potential surface defect is not accurately 
predicted, then it may be assumed that the potential 
defect constitutes random noise and may be disre- 
garded, 

[0025] In order to track a potential surface defect 
from frame to frame, the defect tracking subsystem 30 
requires at least two resulting image frames from the 
anomaly detection subsystem 18, As will be more fully 
described below, a first resulting image frame wilj yield 
potential surface defects which were captured in the 
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second set of image data and a second resulting image 
frame will yield potential surface defects which were 
captured in a third set of image data As will be apparent 
to one skilled in the art, the second set of image data 
and the third set of image data serve as input to the 5 
anomaly detection subsystem 18, thereby resulting in 
the second resultant image frame received from the 
anomaly detection subsystem 18 A more detailed 
description of the defect tracking subsystem 30 is pro- 
vided in re fatten to Figures 4A and 4B 10 
[0026] First, a dilation operation 94 as is wel! known 
in the art is applied to a first resultant image frame 
received from the anomaly detection subsystem 28. in 
this case, the dilation operation merges anomalies that 
are within close proximity of each other (e g , within a 15 
couple of pixels). As a result the image processing 
associated with a subsequent connectivity operation Is 
reduced. 

[0027] The Image frame is then overlaid with a grid 
96 which divides the larger anomalies into smaller so 
anomalies. For instance, a 30x30 pixei grid may be used 
to divide up the larger anomalies As will be more fully 
explained below, this operation helps to accurately pre- 
dict the state of a vector that represents the anomaly 
(which would otherwise be very sensitive to defect 25 
shape changes) and to accurately identify certain struc- 
tural features (e g , a door handle on a side body panel) 
on the vehicle body 

[0028] The amount of potential surface defect data 
is further reduced through the use of a connectivity or 30 
"blobbing" operation 98 Rather than processing a col- 
lection of individual pixel data, the connectivity opera- 
tion forms shaped objects from the related anomaly 
pixels. Tq do so, this operation connects pixels that are 
similar and within one pixel distance from each other, 35 
The connectivity operation is performed within the 
30x30 pixel grid If a "blob" extends outside the grid, 
then it becomes multiple blobs 

[0029] Since the image frame received from the 
anomaly detection subsystem 28 encapsulates image 40 
data from two different image frames, each potential 
surface defect appears twice in the image frame 
received from the anomaly detection subsystem 18 A 
pair elimination operation 1 00 is therefore used to elim- 
inate half of each paired blobs, To do so, a polarity 4s 
check is applied to each set of blobs to eliminate at least 
one of the blobs For instance, a bounding box is drawn 
around each of the blobs The gray level deviation within 
the bounding box is computed for the most recent image 
frame and the previous image frame. If the deviation so 
from the latest image frame is greater than the deviation 
in the previous image frame (and. some predefined 
value), then the blob is determined to be from the latest 
image frame Lastly, the blob from the iatest image 
frame is retained whereas the blob from the previous 55 
image frame is discarded. Accordingly, each remaining 
blob corresponds to potential surface defects which 
were captured in the second set of image data 34 



received by the anomaiy detection subsystem 18 
[0030] Next, a centroid is calculated 102 for each 
remaining blob The centroid will be used by the tracking 
operation 1 06 to predict the position of the potential sur- 
face defect from frame to frame In addition, a compact- 
ness factor (i.e., area/perimeter 2 ) is computed for each 
blob Thus, the compactness factor describes the shape 
of the blob Each blob may be characterized by a state 
vector, where the state vector is defined as a list of nor- 
malized attributes that describe the location and shape 
of an anomaly, It is envisioned that other morphological 
features (eg, perimeter; area, a region of interest, etc ) 
may also be computed and used as attributes to defined 
the state vector for each blob The normafization is nec- 
essary because the information associated with loca- 
tion would have a different range compared with the 
information associated with the shape. These normal- 
ized set of values for location and shape attributes 
describe each blob A state vector for each blob in the 
image frame is then passed atong to the tracking oper- 
ation 106 

[0031] Referring to Figure 4B, the tracking opera- 
tion 1 06 receives a state vector for each blob and tracks 
the state of each vector ss it moves within the reference 
frame of the imaging device The match between the 
predicted state vector and its "true" state is quantified by 
the probability of both being the same Since the geo- 
metric configuration for each blob is relatively invariant 
between frames, the compactness factor is used to 
identify a blob from one frame to the next frame, The 
centroid for each blob is then used track the location of 
a bfob from frame to frame, thereby determining a pre- 
dicted location probability for the blob 
[0032] ] n order to predict the position of the poten- 
tial surface defect, the tracking operation 1 06 must first 
compute the velocity at which the defect is travelling in 
relation to the imaging devJce. For instance, if the 
inspection surface of the workpiece is substantially pla- 
nar, then the potential surface defect is travelling at the 
same rate (in relation to the imaging device) as the 
workpiece is being moved by the conveyor system 
Since the workpiece is being moved by the conveyor 
system at a known constant velocity, the position of or 
distance traveled by the potential surface defect can be 
predicted from frame to frame.. As will be apparent to 
one skilled in the art, the time occurring between frames 
Is known because the imaging device Is capturing 
image data at a known frequency. 
[0033] However, for an inspection surface which is 
not substantially planar, the rate at which the potential 
surface is travelling in relation to the imaging device 
does not correspond to the velocity of the conveyor sys- 
tem, Due to the contour of the inspection surface 120, a 
potential surface defect 122 will appear to be moving 
faster (or slower) than the rate of the conveyor system 
124 as shown in Figure 5, Therefore, the defect tracking 
subsystem 30 utilizes model data indicative of the spa- 
tial relationship between the inspection surface of the 
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workpiece and the observation plane of the Imaging 
device in order to determme the velocity of the potential 
surface defect in relation to the imaging device 
[0034] A CAD modef of an automotive vehicle body 
provides the spatial reiationship between the inspection 5 
surface of the workpiece and the observation plane of 
the Imaging device. A typical CAD model of a vehicle 
body is constructed using 5 000 to 10,000 facets For 
pur poses of the present invention, the vehicle body 1 30 
is modefed using approximately 30 facets as shown in w 
Figure 6 Although the invention is not limited to a sim- 
plified model of the vehicle body, rt is readily understood 
that a 30 facet model provides an adequate approxima- 
tion of the vehicle body as well as transiates into a man - 
ageable amount of model data Model data for each rs 
facet is then used to determine the spatial relationship 
between the observation plane of the imaging device 
and the inspection surface of the workpiece . 
[0035] As the workpiece passes through the obser- 
vation plane of the imaging device, the tracking opera- 20 
tion 106 accesses the model data for the facet- which 
corresponds to the portion of the surface being 
inspected by the inspection system m In this way, the 
tracking operation 106 is abfe to transform the velocity 
of the workpiece into a velocity at which the potential 25 
surface defect is travelling in relation to the imaging 
device The tracking operation 106 then uses this veloc- 
ity to predict the pixel position of the potential surface 
defect from frame to frame Depending on the accuracy 
of this prediction, the tracking operation 1 06 assesses 30 
whether a potential surface defect constitutes a defect 
in the surface of the workpiece 

[0036] Of course, this velocity transformation proc- 
ess must be done when the inspection surface corre- 
sponds to a new facet on the vehicle body. It is further 35 
envisioned that the surface inspection system 10 is 
operative to access different model data for different 
types of vehicle bodies, In addition, the frequency of the 
Imaging device may be synchronized with the vefocity of 
the conveyor system, thereby further simplifying the 40 
velocity transformation process 

[0037] To assess what constitutes a defect, the 
tracking operation 106 computes a tracking strength for 
each potential surface defect as it moves between 
Image frames., The tracking strength is based on the 45 
actual location of the potential surface defect relative to 
the predicted location for that potential surface defect 
and the similarity in compactness between the state 
vectors representing the potential surface defects. Fig- 
ure 7 illustrates how a numeric value is assigned to so 
each pixel in close proximity of the pixel 112 which cor- 
responds to the predicted location of the potential sur- 
face defect If the actual location of the potential surface 
defect 114 fails within one of these pixels, then that 
numeric value is used as a basis for the tracking ss 
strength Because any particular potential surface 
defect may occur in three or more image frames, the 
tracking operation further maintains a cumulative track- 



ing strength value for each potential surface defect This 
cumulative tracking strength value can then be com- 
pared to some predetermined threshold value it is envi- 
sioned that the threshold value Is based on the number 
of times the position for any one potential surface defect 
was predicted within the viewing area of the imaging 
device Finally, each potential surface defect can be 
classified as a surface defector not based on whether 
its cumulative tracking strength exceeds the threshold 
value A surface defect may also be characterized 
based on its cumulative tracking strength and/or the 
number of observations within the viewing area of the 
imaging device. 

[0038] Returning to Figure 4B, the defect tracking 
subsystem 30 includes at least two additional post- 
processing operations. First , a mapping operation 108 
as is well known in the art can be used to map any iden- 
tified surface defects back onto a model of the vehicle 
body A camera model provides a numericaf formulation 
that enables accurate description In the physical world 
of a pixel on the imaging device in this case the cam- 
era model incorporates the physical principles that dic- 
tate the formation of an image in the imaging device 
Using this camera model, a ray is drawn through a 
potential surface defect such that it passes through a 
surface on the vehicle body as shown in Figure 8 
[0039] Modei data for each facet is translated along 
with the vehicle body. This trans iati on is dictated by the 
measurement of position of the vehicle body Intersec- 
tion of this ray with a facet provides the location of the 
defect on the vehicle body A vector summation of this 
ray with the translation vector associated with a facet on 
the vehicle body provides a method for accurately pre- 
dicting the position of the defect after car translation 
Again, the camera model is used to calculate the posi- 
tion of the defect in the image after translation. The 
accuracy of the prediction depends on the accuracy of 
the facet model data and thus the translation vector. It 
should be noted that a velocity calculated using the 
above procedure is not limited to a single direction of 
travel for a surface defect along a non -planar surface of 
the vehicle body. Identified surface defects can also be 
displayed on a CRT or other type of display device 
which is associated with the computing device 18 of the 
surface inspection system 10. 

[0040] A masking operation 1 10 is then used to fil- 
ter out any known structural features of the vehicle body 
In this case, since the structural features (e,g , a door 
handle on a side body panel) are identified as surface 
defects by the surface inspection system, the masking 
operation 1 1 0 ensures that these features are not misi- 
dentified as surface defects. 

[0041] While the invention has been described in its 
presently preferred form, it will be understood that the 
Invention is capable of modification without departing 
from the spirit of the invention as set forth in the 
appended claims 
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Claims 

1., A surface inspection system for detectmg defects 
on a surface of a workpiece (25 120), in particular 
for detecting defects on a surface of a vehicle body 5 
(14) within a motor vehicle manufacturing system, 
characterized by: 4. 

at least one diffused light source (22) for emit- 
ting an elongated line of iight (23) onto the sur- io 
face of the workpiece (25, 120); 
a movable member (26), in particular a con- 
veyer system (1 2), for translating the workpiece 
(25, 120) in relation to said light source (22); 5, 
an imaging device (24) for capturing two or 15 
more sets of image data (32, 34) representa- 
tive of a portion of the surface of the workpiece 
(25, 120), said imaging device (24) having a 
fieid of observation and being positionable at a 6„ 
vantage point such that the Jine of light (23) is 20 
withtn the field of observation; 
a data structure for storing model data indica- 
tive of a spatial relationship between said por- 
tion of the surface of the workpiece (25, 120) 
and the observation plane of said imaging 25 
device (24) and; 

an anomaly detection subsystem (28) adapted 
to receive at least a first set of image data (32) 
and a second set of image data (34) from said 
imaging device (24), said anomaly detection so 
subsystem (28) identifying at teast one poten- 
tial surface defect (1 T4 122) in said first set of 
image data (32) and said second set of image 
data (34); and 7, 
a tracking subsystem (30) connected to said 35 
data str ucture and said anornary detection sub- 
system (28) for tracking the potential surface 
defect (114, 122) from said first set of image 
data (32) to said second set of image data (34) 
using said model data, thereby assessing if the 40 
potential surface defect (114, 122) constitutes 
a defect In the surface of the workpiece (25, 
120). 

2 . The surface inspection system of Claim 1 , charac- 45 
terized in that said tracking subsystem (30) opera- 
tive to transform the velocity (v) of the workpiece 
(25) Into a velocity at which a potential surface 
defect (1 14, 122) is moving in relation to said imag- 
ing device (24) through the use of said model data, 50 
and to predict a location of the potential surface 
defect (114, 122) from said first set of image data 
(32) to said second set. of image data (34) through 
the use of said velocity (v) 
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3, The surface inspection system of Claim 1 or 2, 
characterized in that the workpiece (25, 120) is 
modeled using a plurality of facets, where each 



facet represents a portion of the surface of the 
workpiece {25, 120), and said model data is indica- 
tive of the spatial relationship between each facet of 
the workpiece (25 120) and the observation plane 
of said imaging device (24) 

The surface inspection system of Claim 3, charac- 
terized in that said tracking subsystem (30) is oper- 
ative to access model data which corresponds to 
the portion of the surface of the workpiece (25, 1 20) 
being inspected by the surface inspection system 
(10). 

The surface inspection system of any of Claims 1 to 

4, characterized by means for adjusting an intensity 
of said light source (22) based on the cofor of the 
surface of the workpiece (25) 

The surface inspection system of any of Claims 1 to 

5, characterized in that said anomaiy detection sub- 
system (28) includes: 

a subtraction module (36) receiving said first 
set of image data (32) and said second set of 
image data (34) and operative to generate 
image data (38) indicative of the differences 
between said first set of image data (32) and 
said second set of image data (34); and 
a thresholding module (40) receiving the image 
data (38) from said subtraction module (36) 
and operative to generate a binary representa- 
tion (42) of said image data. 

The surface inspection system of any of Claims 1 to 

6, character ized in that said anomaly detection sub- 
system (28) further includes: 

an edge detection module (52) receiving said 
first set of image data (32) and said second set 
of image data (34) and operative to identify the 
edges of the line of light (23) in each set of 
image data (32, 34); 

a second thresholding module (58) receiving 
the first and second sets of image data (54 f 56) 
from said edge detection module (52) and 
operative to generate a binary representation 
for each set of image data; 
a dilation module (60) receiving said first and 
second sets of image data from said second 
thresholding module (58) and operative to 
thicken the edges of the line of light (23) in 
each set of Image data; and 
a mask formation module (110) receiving said 
first and second sets of image data (62, 64) 
from said dilation moduie (60), said mask for- 
mation module (1 1 0) operative to combine said 
first set of image data (62) with said second set 
of image data (64) into a resultant set of image 
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data, using an OR operation (66) and to invert 
the resultant image data (68) using a NOT 
operation (70), thereby forming a mask (72} for 
filtering out a portion of the noise in the Image 
data 5 

8, The surface inspection system of any of Claims 1 to 
7, characterized in that said anomaly detection sub- 
system (28) further includes: 

10 

an anomaly Identification module receiving 
image data (42) from said thresholding module 
(40) and a mask (72) from said mask formation 
module and operative to apply said mask (72) 
to said image data (42) using an AND opera- ts 
tion (74), thereby filtering out a portion of the 
noise in the image data, 

9. A method of detecting defects in a surface of a 
workpiece (25, 120) using a surface inspection sys- 20 
tern (10), comprising the steps of: 

emitting at least one line of light (23) onto the 
surface of the workpiece (25, 120); 
passing the surface of the workpiece (25, 120) 25 
through the Ime of light (23); 
capturing a first set of image data (32) repre- 
sentative of a portion of the surface of the work- 
piece (25, 120) using an imaging device (24), 
said imaging device (24) having a plane of 30 
observation and being positi enable at a van- 
tage point such that the line of light (23) is 
within its piane of observation; 
providing model data Indicative of the spatial 
relationship between said portion of the surface 35 
of the workpiece (25, 120) and the observation 
piane of said imaging device (24); 
capturing a second set of image data (32) rep- 
resentative of said portion of the surface of the 
workpiece (25, 120) using said imaging device 40 
(24); 

identifying at least one potential surface defect 
(114, 122) in said second set of image data 
(34); 

capturing a third set of image data representa- 45 
tive of said portion of the surface of the work- 
piece (25, 120) using said imaging device (24); 
and 

tracking the potential surface defect {114, 122) 
from said second set of image data (32) to said so 
third set of image data using said model data, 
thereby assessing if the potential surface 
defect (114, 122) constitutes a defect in the 
surface of the workpiece (25, 120) 

55 

10, A method of detecting defects in a. surface of a 
workpiece (25, 120), comprising the steps of: 



emitting at ]east one line of light (23) onto the 
surface of the workpiece (25, 1 20); 
passing the surface of the workpiece (25, 120) 
through the line of light (23); 
capturing a first set of image data (32) repre- 
sentative of a portion of the surface of the work- 
piece (25, 120) using an imaging device (24), 
said imaging device (24) having a plane of 
observation and being positionable at a van- 
tage point such that the line of light (23) is 
withtn its plane of observation; 
identifying at least one potential surface defect 
(1 1 4, 1 22) in said first set of image data (32); 
determining a velocity (v) for the workpiece (25, 
120); 

transforming the velocity (v) of the workpiece 
(25, 120) into a velocity {v') at which potential 
surface defects (114, 122) are. moving in rela- 
tion to said imaging device (24); 
capturing a second set of image data (34) rep^ 
resentative of said portion of the surface of the 
wor kpiece (25, 120) using said imaging .device 
(24); and 

tracking the potential surface defect (114, 122) 
from said first set of image data (32) to said 
second set of Image data (34) using the veloc- 
ity (v 1 ) at which potentiaf surface defects (1 14, 
122) are moving in relation to said imaging 
device (24), thereby assessing if the potential 
surface defect (1 1 4, 122) constitutes a defect in 
the surface of the workpiece (25, 120) 

11., The method of Claim 9 or 10, further comprising the 
step of adjusting an intensity of a light source, emit- 
ting said at least one line of fight (23) based on the 
color of the surface of the workpiece (25, 120). 

12, The method of any of Claims 9 to 1 1, wherein the 
step of identifying at least one potential surface 
defect (1 1 4, 1 22) further comprises: 

subtracting (36) said first set of image data (32) 
from said second set of image data (34) to form 
an intermediary set of image data (38), the 
intermediary set of jmage data (38) indicative 
of potentiat surface defects (1 1 4, 1 22) in at 
least said second set of image data (32) 

13:, The method of Claim 12, which further comprises 
generating a binary representation (42) of the inter- 
mediary set of image data (38) 

14, The method of Claim 12 or 13, which further com- 
prises the step of fjftering out at least a portion of 
noise in said first and second set of image data, 
where the noise is caused by a variation in the vis- 
cosity of the paint on the surface of the workpiece 
(25,120) 
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15 . The method of Claim 14, wherein the step of filter- 
ing out at least a portion of noise further comprises: 

providing a copy of said first set (32) and said 
second set (34) of image data; 5 
identifying (52) the edges of the line of light (23) 
fn each copy of image data; 
combining the copy of said first set of image 
data with the copy of said second set of image 
data into a resultant set of image data toy using io 
an OR operation (66); 

inverting said resultant set of image data using 
a NOT operation (70), thereby forming a mask 
(72) for filtering out a portion of the noise; and 
applying said mask (72) to said intermediary 15 
set (42) of image data using an AND operation 
(74), thereby filtering out a portion of the noise 
in the image data 

16,, The method of any of Claims 9 to 15, further com- so 
prising: 

modeling the workpiece (25, 120) using a plu- 
rality of facets, where each facet represents a 
portion of the surface of the workpiece (25, 2s 
*20); 

providing mode! data indicative of the spatiai 
relationship between each facet of the work- 
piece (25, 120) and the observation plane of 
said imaging device (24); and 30 
tracking the potential surface defect (114, 122) 
using model data which corresponds to the 
portion of the surface of the workpiece (25, 
120) being inspected by the surface inspection 
system (10). 35 

17- The method of any of Claims 9 to 16, wherein the 
step of tracking the potential surface defect further 
comprises: 

40 

determining a first location of the potential sur- 
face (114, 122) defect in a reference frame for 
said imaging device (24), where the first loca- 
tion is derived from said second set (34) of 
image data; 45 
determining a velocity (v) for the workpiece (25, 
120); 

transforming the velocity (v) of the workpiece 
(25, 120) into a velocity (v 1 ) at which potential 
surface defects (114, 122) are moving in rela- so 
tion to the said imaging device (24) through the 
use of said model data; 

predicting a tocation of the potential surface 
defect (114, 122) in said third set of image data 
by using the velocity (v 1 ) for potential surface 55 
defects (114,122); 

determining a second location of the potential 
surface (114, 122) defect In the reference 



frame of said imaging device (24), where the 
second location is derived from said third set of 
image data; and 

comparing the predicted location of the poten- 
tial surface defect (114, 122) with said second 
location of the potential surface defect (114, 
122); thereby tracking the potential surface 
defect (114/122) 
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